Meta-Modeling in Multiobjective Optimization

In many practical engineering design and other scientific optimization problems, the objective function is not given in closed form in terms of the design variables. Given the value of the design variables, the value of the objective function is obtained by some numerical analysis, such as structural analysis, fluidmechanic analysis, thermodynamic analysis, and so on. It may even be obtained by conducting a real (physical) experiment and taking direct measurements. Usually, these evaluations are considerably more time-consuming than evaluations of closed-form functions. In order to make the number of evaluations as few as possible, we may combine iterative search with meta-modeling. The objective function is modeled during optimization by fitting a function through the evaluated points. This model is then used to help predict the value of future search points, so that high performance regions of design space can be identified more rapidly. In this chapter, a survey of meta-modeling approaches and their suitability to specific problem contexts is given. The aspects of dimensionality, noise, expensiveness of evaluations and others, are related to choice of methods. For the multiobjective version of the meta-modeling problem, further aspects must be considered, such as how to define improvement in a Pareto approximation set, and how to model each objective function. The possibility of interactive methods combining meta-modeling with decision-making is also covered. Two example applications are included. One is a multiobjective biochemistry problem, involving instrument optimization; the other relates to seismic design in the reinforcement of cable-stayed bridges. 10.1 An Introduction to Meta-modeling In all areas of science and engineering, models of one type or another are used in order to help understand, simulate and predict. Today, numerical methods Reviewed by: Jerzy Błaszczyński, Poznan University, Poland Yaochu Jin, Honda Research Institute Europe, Germany Koji Shimoyama, Tohoku University, Japan Roman Słowiński, Poznan University of Technology, Poland J. Branke et al. (Eds.): Multiobjective Optimization, LNCS 5252, pp. 245–284, 2008. c © Springer-Verlag Berlin Heidelberg 2008 246 J. Knowles and H. Nakayama make it possible to obtain models or simulations of quite complex and largescale systems, even when closed-form equations cannot be derived or solved. Thus, it is now a commonplace to model, usually on computer, everything from aeroplane wings to continental weather systems to the activity of novel drugs. An expanding use of models is to optimize some aspect of the modeled system or process. This is done to find the best wing profile, the best method of reducing the effects of climate change, or the best drug intervention, for example. But there are difficulties with such a pursuit when the system is being modeled numerically. It is usually impossible to find an optimum of the system directly and, furthermore, iterative optimization by trial and error can be very expensive, in terms of computation time. What is required, to reduce the burden on the computer, is a method of further modeling the model, that is, generating a simple model that captures only the relationships between the relevant input and output variables — not modeling any underlying process. Meta-modeling , as the name suggests, is such a technique: it is used to build rather simple and computationally inexpensive models, which hopefully replicate the relationships that are observed when samples of a more complicated, high-fidelity model or simulation are drawn.1 Meta-modeling has a relatively long history in statistics, where it is called the response surface method, and is also related to the Design of Experiments (DoE) (Anderson and McLean, 1974; Myers and Montgomery, 1995). Meta-modeling in Optimization Iterative optimization procedures employing meta-models (also called surrogate models in this context) alternate between making evaluations on the given high-fidelity model, and on the meta-model. The full-cost evaluations are used to train the initial meta-model, and to update or re-train it, periodically. In this way, the number of full-cost evaluations can often be reduced substantially, whilst a high accuracy is still achieved. This is the main advantage of using a meta-model. A secondary advantage is that the trained meta-model may represent important information about the cost surface and the variables in a relatively simple, and easily interpretable manner. A schematic of the meta-modeling approach is shown in Figure 10.1. The following pseudocode makes more explicit this process of optimization using models and meta-models: 1. Take an initial sample I of (x,y) pairs from the high-fidelity model. 2. From some or all the samples collected, build/update a model M of p(y ∈ Y |x ∈ X) or f : X → Y . 1 Meta-modeling need not refer to modeling of a computational model ; real processes can also be meta-modeled. 10 Meta-Modeling in Multiobjective Optimization 247 parameters objectives $$$